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Reasoning is considered to be an important proficiency in national mathematics curricula 
both in Australia (ACARA, 2014) and Malaysia (MOE, 2013). However, the nature of 
reasoning that supports learning and problem solving in mathematics is an area that 
requires further study (Schoenfeld, 2013). In this study we explored the link between 
Scientific Reasoning Skills (SRS) and mathematics problem solving performance among a 
cohort of Malaysian students. As expected, there was a positive relationship but the level of 
correlation between these two variables was moderate. Although the High -Achievement 
group performed significantly better than their peers in the Low-Achievement group in 
their solution outcomes, overall, all students exhibited low-levels of SRS. These findings 
suggest that while SRS could play a role in problem solving, components of SRS need 
further analysis in order to better explain how reasoning in science could facilitate problem 
solving processes. 


Introduction 


With increasing attention to logical arguments and justifications in mathematics, the 
study of reasoning that underpins these processes constitutes an important area of research 
(Schoenfeld, 2013; Santos-Trigo & Moreno-Armella, 2013). In this study, we focussed on 
reasoning that is associated with the development of arguments and justifications in the 
context of problem solving. Data on reasoning and how that reasoning is used by students 
during the course of their solution search was expected to deepen current understandings 
about the construction of arguments and generation of justifications by learners. In this 
study, we generate data relevant to this issue by analysing scientific reasoning processes 
that students could activate during their mathematical problem-solving attempts. 


Literature review 


Reasoning in Mathematics 

Current research emphasises the importance of students engaging in reasoning in all 
strands of school mathematics (National Council of Teachers of Mathematics, 2009; Bieda 
et ai, 2013; Santos-Trigo & Moreno-Armella, 2013; Stylianides et al., 2008). Ball and 
Bass (2003, p. 42) argued that ‘mathematical reasoning is inseparable from knowing 
mathematics with understanding.’ Several scholars have elaborated on the connection 
between learning mathematics with understanding and reasoning. Lakatos (1976) noted 
that complete mathematical understanding includes the engaging processes of thinking, in 
essence doing what makers and users of mathematics do: framing and solving problems, 
patterns recognition, making conjectures, examining constraints, making inferences from 
data, abstracting, inventing, explaining, justifying, challenging, and so on. This observation 
about understanding and reasoning was recently reaffirmed by Schoenfeld (2013) when he 
commented that one variable seemed to have strongest impact on students learning - the 
amount of time students spent in explaining and justifying their ideas. While definitions 
used in the context reasoning differ, they deal in one way or another with a broad range of 
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thinking skills involving arguments that drive the evolution of solutions to mathematical 
problems. In this sense, there are aspects of mathematical problem solving that could draw 
on scientific reasoning (Hand, Prain & Yore, 2001). 

Scientific Reasoning Skills (SRS) 

The range of SRS that students bring to learning and problem solving can be expected 
to assist them in making progress in multiple ways. SRS ‘encompasses the reasoning and 
problem-solving skills involved in generating, testing and revising hypotheses or theories, 
and in the case of fully developed skills, reflecting on the process of knowledge acquisition 
and knowledge change that result from such inquiry activities’ (Morris, Croker, Masnick & 
Zimmerman, 2012, p. 65). Scientific reasoning differs from other skills in that it requires 
additional cognitive resources as well as an integration of cultural tools. Further, scientific 
reasoning emerges from the interaction between internal factors (e.g., cognitive and 
metacognitive development) and cultural and contextual factors. According to Lawson 
(2004), scientific reasoning pattern is defined as a mental strategy, plan, or rule used to 
process information and derive conclusions that go beyond direct experience. In a similar 
vein. Hand, Prain and Yore (2001) argued that scientific reasoning involves the ability to 
construct powerful arguments for learners’ actions. Thus, SRS is related to cognitive 
abilities such as critical thinking and reasoning that assist students in producing knowledge 
during problem solving through evidence-based reasoning. Given the connectedness 
between knowledge generation via arguments and reasoning that gird these arguments, 
students with higher levels SRS could be expected to be superior problem solvers. 

Conceptual framework 

Lawson’s (2000) Classroom Test of Scientific Reasoning provided the framework to 
guide the analysis and interpretation of data in the present study. 

Lawson ’s Classroom Test of Scientific Reasoning 

In this study, we focus on a set of domain-general reasoning skills that are commonly 
needed for students to make progress with mathematical problems which includes 
exploring a problem, formulating arguments, manipulating and isolating variables, and 
observing and evaluating the production of new information. Lawson’s Classroom Test of 
Scientific Reasoning (LCTSR) provides a theoretical lens for assessing a range of SRSs. 
The test was designed to examine a set of general reasoning ability dimensions which are 
crucial for the solution of problems in STEM including conservation of matter and volume, 
proportional reasoning, control of variables, probability reasoning, correlation reasoning, 
and hypothetical-deductive reasoning. The validity of the LCTSR had been established by 
several studies (e.g. Lawson, Bank & Lovgin, 2007). 

LCTSR allows for the observation of three levels of reasoning: concrete, transitional 
and formal operational reasoning. The concrete operational reasoning refers to thinking 
pattern that enable one to understand concepts and statements that make a direct reference 
to familiar actions and observable objects, and can be explained in terms of simple 
associations (for example, all squares are rectangles but not all rectangles are squares). 
Students, at this level of reasoning, are also able to follow step-by-step instructions 
provided each step is completely specified (for example, solving two linear equations). 
Students are also able to relate his/her viewpoint to that of another in familiar situations 
(for example, students respond to difficult mathematical problems by applying a related 
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correct rule). At this stage, students are unconscious of his/her own reasoning patterns, 
inconsistencies among various statements he/she makes, or contradictions with other 
known facts. 

In contrast to concrete reasoning, formal reasoning patterns enable students to 
construct possible explanations as a starting point for reasoning about a causal situation. 
They can reason in a deductive manner to test their hypotheses. In other words, they can 
postulate causal factors, deduce the consequences of these possibilities and then 
empirically verify which of those consequences, in fact, occurs. Lawson (1978) categorised 
students at this stage as ‘reflective thinkers’. For example, in solving mathematical 
problem, students’ reasoning can be initiated with development of representations, use of 
symbols and planning a course of action. 

The transitional operational stage is where students remain confined to concrete 
thinking or are only capable of partial formal reasoning. For example, proportional 
reasoning is the ability to compare ratios or develop arguments about the equality between 
two ratios. At concrete operational stage, students are not aware of ratio dependence and 
seek solutions by guessing. At the transitional stage, students are aware of objective 
dependence and seek solutions by estimation and later calculation, but assume that the 
change in one quantity produces the same change in the other quantity. In the formal stage, 
proportionality is discovered and applied to obtain correct solutions. Clearly, in all three 
levels of reasoning, students generate qualitatively different types of information that are 
driven by arguments and justifications. 

Purpose of the Study 

The review of literature indicates that reasoning skills are transferable across science 
and mathematics (Hand, Prain & Yore, 2001), and that we could learn a great deal about 
the role of reasoning in mathematical problem solving by examining potential links 
between the two (Lehrer & Schauble, 2000). The purpose of the study was to identify the 
levels of SRS attained by a cohort of upper Malaysian secondary school students (16-17- 
year-olds) and examine the impact of SRS on their mathematical problem-solving 
performance. We sought data relevant to the following three research questions: 

1. What are the levels of SRS among upper secondary school students? 

2. Is there a relationship between SRS and mathematics problem-solving 
performance? 

3. Does achievement level of students affect their SRS and mathematics problem- 
solving performance? 


Methodology 


Design 

This study employed a blend of descriptive and correlational research design as our 
interest was to generate information about the relationship between one independent 
(Achievement level) and two dependant variables (SRS and Mathematics Problem 
Solving). 
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Participants 

A total of 351 students from 14 Malaysian secondary schools participated in the 
present study. Participants in this study were upper secondary school students or Year 11 
students (16-17-year-olds). Participants were assigned to High or Low achievement groups 
on the basis of their performance in a centralised Malaysian examination, Lower 
Secondary Evaluation Examination (LSEE). The High- achievement group (Grades A or B 
in LSEE) comprised Science stream students (n = 98) and the Low-achievement group 
(Grades C or D in LSEE) were Non-Science stream students (n = 253). This is based on the 
Malaysian Education Evaluation System in placing students into Science and Non-Science 
streams. Under this system, students needed to score high marks in mathematics in order to 
go into the Science Stream in comparison to their peers in the Non-Science stream. 

Tasks 

This section provides details of two tasks that were used in this study. As discussed 
earlier, there were two tests used in order to generate scores for two dependent variables: 
Test 1 - Scientific Reasoning Test (SRT); Test 2 - Mathematics Problem-solving Test 
(MPST). 

Task 1 - Scientific Reasoning Test (SRT) 

The SRT was used to measure the students’ level of SRS. It has been adapted from 
LCTSR (Lawson, 2000). We wanted to determine the internal consistency of the items 
which involved the generation of inter-item correlation matrix and computing of Kuder- 
Richardson 20 internal-consistency reliability coefficient. The final test had a KR-20 
coefficient of 0.856. The test consisted of 12 paired items and was designed in a ‘two- 
stage’ multiple-choice format to illustrate problem scenarios. With each scenario, the first 
question focuses on the scenario content, while the second question asks for reason as to 
why the first answer is correct. Each answer for the first question has a corresponding 
reason in the second question. 

For example, in one of the tasks, students’ reasoning about conservation of volume was 
evaluated. Firstly, students have to think based on their experience or previous knowledge 
about where the water will rise to when the glass marble is put into cylinder. Then, 
students have to justify as to why the water rose at that level. This involves students 
applying the conservation reasoning to perceptible objects and properties. Making 
prediction and giving explicit explanation are important to successful completion of this 
item. Prediction, explanation and the generation of relevant new information are important 
processes of mathematical problem solving. Thus, we argue that these reasoning skills 
would contribute to the solution outcome of mathematical problems. 

Scoring Rubric for SRT 

The range of scores of SRS level is 0-12 which decomposes into three levels as 
suggested by Lawson: 0-4 (Concrete); 5-8 (Transitional); 9-12 (Formal). 

Task 2 - Mathematics Problem Solving Test (MPST) 

The MPST was designed to measure students' mathematics problem-solving 
performance by drawing on SRSs. The test was prepared by a panel of experienced 
mathematics educators, experienced teachers and mathematics curriculum experts. We 
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were concerned to ensure that the solution of the problems necessitated the activation of 
one or more levels reasoning that was postulated in the framework of SRS. The items for 
the test were selected from a pool of resources such as textbooks, reference books and 
examination papers. The test consisted of 40 items that covered all core mathematical 
strands in the Malaysian Mathematics Syllabus (Year 8 - Year 11). The reliability of the 
test was established by following a process that was similar to SRT. The reliability index 
for MPST was 0.895. 

For example, Item 18 required students to work out the perimeter of an irregular shape 
that was located within a rectangle. The solution required students to generate arguments 
about different ways to determine the perimeter and test their hypothesis. Students were 
categorised as having concrete reasoning level if they needed reference to familiar actions, 
objects, and descriptive properties. At this level, their reasoning was initiated with 
observations and step-by-step moves. For Item 18, students may only show a superficial 
understanding of concepts of perimeter, area of a rectangle and a circle without any way 
linking these to solving the problem. Students were categorised as having formal reasoning 
level if they can be initiated with possibilities, used symbols to express ideas, planned a 
lengthy procedure given the overall goal while being critical of his/her own reasoning 
patterns. In this case, students may systematically plan to find perimeter of the irregular 
shaded region. This will involve finding the curve length of a semicircle and a quadrant 
using the formulae for the area and using the given information of the radius length. 
Students were categorised as having a transitional reasoning level if they remained 
confined to concrete thinking or are only capable of partial formal reasoning such as they 
only understood and applied concepts of perimeter and area of a rectangle and area of a 
circle in a new context. Students responses for MPST were scored as 1- correct response; 
0- incorrect response. 

Procedures 

There were three phases in the study. The first phase was concerned with the 
development and fine-tuning of MPST. The details are explained in the MPST task section. 
During the second phase, we pilot tested both the tests to allow for familiarisation of the 
data collection processes, to validate the instruments and to establish their reliability. The 
third phase involved the administration of the two tests. Both tests were administered 
during regular mathematics classes. Researchers and classroom teachers assisted with the 
administration of the tests. Students were invited to complete the SRT in the first week of 
their regular mathematics lesson. They were allowed a maximum of 40 minutes for SRT. 
The MPST, a one-hour paper and pencil test, was administered in the second week, again, 
during their regular mathematics lesson. 

Results and Analysis 

Three research questions were of interest to the present study. Data relevant to these 
research questions are presented below. 

Research Question 1: What are the levels of SRS among Upper Secondary School 
Students? 

Table 1 shows the overall level of SRS exhibited by the participating Year 11 students. 
The findings showed that 330 (94%) of the students achieved Level 1 ( concrete ) of SRS, 
20 (5.7%) Level 2 ( transitional ) and 1 (0.3%) Level 3 (formal ). The overall mean level of 
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the SRS was 1.76. This indicates that majority of the participating students were 
functioning at the concrete level of SRS. 


Table 1 
Level of SRS 


SRS Level 

N 

Percentage(%) 

SRS Mean 
Score 

Standard 

Deviation 

Concrete 

330 

94.0 

1.50 

1.18 

Transitional 

20 

5.70 

5.65 

0.81 

Formal 

1 

0.30 

9.00 

- 

Total 

351 

100.00 

1.76 

1.55 


Research Question 2: Is there a Relationship between the SRS and Mathematics 
Problem Solving Performance? 

Overall, the correlation between the SRS and mathematics problem solving 
performance was significant indicating a positive relationship between the two variables [(r 
= 0.593), p<0.05]. The coefficient of correlation (r = 0.593) indicating that there was a 
moderate positive relationship between the SRS and mathematics problem solving 
performance .This suggests that if a student had a high score in SRS, he/she are expected to 
achieve a high score in MPST. 

Research Question 3: Does Achievement Level of Students Effect their SRS and 
Mathematics Problem-Solving Performance? 

A t-test analysis was conducted to compare the mean scores of the overall level of SRS 
for the two Achievement levels (High/Low). Analysis as presented in Table 2 showed there 
were differences in mean overall SRS between High and Low-Achievement groups [t 
(349) = 9.260, p < 0.05]. The mean SRS level for High- Achievement group (mean = 2.99) 
was better than the corresponding score for peers in the Low-Achievement group (mean = 
1.28). However, SRS score for both groups of students was 1.76 (Table 1) suggesting the 
students had acquired concrete reasoning level. 

The total scores for MPST were converted to percentages. Mean percentages for the 
High-Achievement group and the Low Achievement group were 81.02 and 46.86 
respectively (Table 2). The results also showed there were differences in mean MPST 
percentages between the High and Low-Achievement groups [t(349) = 16.789, p < 0.05]. 
The mean MPST percentage score for the High- Achievement group was higher than the 
Low- Achievement group. Taken together, students in the High- Achievement group 
produced higher scores for SRS and MPST than their peers in the Low-Achievement 
group. 
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Table 2 

SRS score and MT score Vs Achievement Group 




Achievement Group 



Dependent Variable 


High 

Low 

t-value 

p- value 



(n= 98) 

(n=253) 



SRT score 

Mean 

2.99 

1.28 

9.260 

** 


SD 

1.66 

1.21 



MPST score 

Mean 

81.02 

46.86 

16.789 

** 


SD 

13.42 

18.32 




**p<0.01 


Discussion 

The study was designed to generate data relevant to issues about relation between the 
level of SRSs and mathematics problem solving among a cohort of Malaysian students. 
The first research question addressed the level of SRS among upper secondary school 
students. We found participants to have acquired moderate levels of SRS. Almost all the 
students (94.0%) were in the concrete reasoning level and others were in the transitional 
(5.7%) and formal (0.3%) reasoning levels. The second research question addressed the 
relationship between SRS and mathematics problem solving. The results indicate that there 
was a moderate positive correlation between the SRS and mathematics problem solving 
ability as measured by the MPST. Data analysis relevant to Research Question 3 showed 
that students in the High-Achievement group performed significantly better than their 
Low- Achievement peers in both the MPST and SRS. Given the positive correlation 
between SRS and MPST, it was expected that the higher MPST scores of particularly the 
High- Achievement group can be attributed to their superior SRS. However, the SRS scores 
for all the students including the High- Achievement group was relatively low suggesting 
they were operating at concrete level. 

Interestingly, the higher SRS scores for the High- Achievement group (in comparison to 
the Low-Achieving Group) is still low in terms of the overall SRS level that they had 
achieved. The mean SRS score for this group was 2.99 which falls well into the concrete 
reasoning level. However, as shown in Table 2, despite relatively low SRT scores for the 
High-Achievement group, the score on MPST for this group was significantly higher 
(mean = 81.02) in comparison to the Low-Achievement group. 

We offer two possible explanations for the above pattern of results. Firstly, it might be 
that students in High- Achievement group were using reasoning and information generating 
strategies that do not involve the use of SRSs, a claim that needs further investigation. A 
second possibility is that concrete level SRS may be sufficient for the solution of the type 
problems that were provided in our MPST. If the latter is indeed the case, the suggestion is 
that we may have to develop more complex and sensitive problems in order to constrain 
students to activate transitional and formal levels of scientific reasoning during their 
solution attempts. In our next phase of this study, we are planning to pursue this 
hypothesis. 

The scores for SRS and problem-solving for students in the Low-Achievement group 
were low in comparison to their high-achieving peers. In the absence of further data about 
how SRSs could foster problem solving in mathematics, it is too early to argue that low- 
achieving students could benefit from instructions to improve their SRSs. We also suggest 
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that the scoring of SRS and MPST needs fine-tuning in order to make the comparison more 
sensitive to the skills underpinning the two variables. 

In our analysis of level of SRS and mathematical problem solving, we did not consider 
the cultural context of the participating students. The students in this study had three types 
of linguistic backgrounds - Malay, Mandarin and Tamil. It would be interesting to explore 
the link between students’ linguistic background, scientific reasoning skills and 
mathematical problem solving outcomes. In the present study, we drew on Lawson’s work 
concerning the three levels of scientific reasoning skills on the assumption that these levels 
would be sufficient in order to capture the multitude of reasoning that could be activated 
during novel mathematical problem solving. As mentioned above, although all students 
were operating at Level 1 of SRS, their performance in MPST, particularly for the High- 
Achievement group, was high. It would seem that students were indeed engaging in 
substantial reasoning when they completed the MPST tasks. 
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